Note on Algorithm Differences Between Nonnegative Matrix Factorization And Probabilistic Latent Semantic Indexing

NMF and PLSI are two state-of-the-art unsupervised learning models in data mining, and both are widely used in many applications. References have shown the equivalence between NMF and PLSI under some conditions. However, a new issue arises here: why can they result in different solutions since they are equivalent? or in other words, their algorithm differences are not studied intensively yet. In this note, we explicitly give the algorithm differences between PLSI and NMF. Importantly, we find that even if starting from the same initializations, NMF and PLSI may converge to different local solutions, and the differences between them are born in the additional constraints in PLSI though NMF and PLSI optimize the same objective function.